Dynamic visual servo control of robots: An adaptive image-based approach

Sensory systems, such as computer vision, can be used to measure relative robot end-effector positions to derive feedback signals for control of end-effector positioning. The role of vision as the feedback transducer affects closed-loop dynamics, and a visual feedback control strategy is required. Vision-based robot control research has focused on vision processing issues, while control system design has been limited to ad-hoc strategies. We formalize an analytical approach to dynarnic robot visual servo Control systems by first casting position-based and imagebased strategies into classical feedback control structures. The image-based structure represents a new approach to visual servo control. which uses image features (e.g., image areas, and centroids) as feedback control signals, thus eliminating a complex interpretation Step (i. ., interpretation f image features to derive world-space coordinates). Image-based control presents formidable engineering problems for controller design, including coupled and nonlinear dynamics, kinematics, and feedback gains, unknown parameters, and measurement noise and delays. A satisfy these requirements. model Leference adaptive controller (MRAC) is designed to 1 . Introduction Flexible robotic systems should provide acapability to automatically modify positions and trajectories to accommodate changes in the task or task environment. Such flexibility is achievable only through the integration of sensors into robotic systems, and visual sensing, in particular, provides fundamental capabilities in the adaptation of robots to both unstructured and precision tasks. The promise of fast vision systems and special purpose high-speed optical sensors would permit the synthesis of dynamic "visual servo control systems." In this paper, we describe a strategy which integrates visual sensing with robot movements, and will emphasize the design of such a "visual servo control system" in order to achieve stable and predictable system dynamics. Vision systems can be used as transducers to measure the relative positions between robot end-effectors and objects in the robot's work space. The interpretation of two dimensional image information to estimate positions normally requires the 'This research has been supported in part by the Westinghouse Electric Corporation and in part by the Robotics Institute of Carnegie-Mellon University. extraction of quantitative image features which are related to stored object models. The positional error signal, between reference and measured positions, may be used to design a "position-based'' visual servo control system'. In our research we have explored an alternative strategy*. Image features which are monotonically related to spatial position may be used as a basis for control in place of position estimates. These features typically include structural components of the image such as points, lines, and areas, as well as quantitative parameters attached to them. Such an "image-based'' visual servo control strategy may offer advantages in terms of reduced delay and estimation noise, and provide a convenient means for robot task training. However, image-based control requires special attention to control strategies which accommodate position-to-feature relationships. In dynamic visual servo control systems the role of computer vision as the measurement process affects the overall system dynamics, and a visual feedback controller is required for stability and to achieve a suitable transient response. The linearity, noise properties, coupling, and computational delays of this measurement process become essential considerations for controller design. Formal analysis and design of feedback controllers for visual servoing, in terms of well-defined control theory, has not appeared in literature except for a simple case3. Visual servo controllers have been designed using ad-hoc strategies4. In our research, we have organized previously implemented and proposed visual servo control strategies into clearly defined feedback control structures, and clarify the requirements placed on the visual servo controller5. In position-based approaches to visual servo control, the image interpretation step can be complex and affect control system design and performance by adding additional time delays, and introducing measurement noise resulting from inaccuracies of the object and transduction modeling. Image features are often continuous functions of object position, and for fixed ranges of control, provide sufficient information for robot control without carrying out the interpretation process. An image-based system is described in more detail below. The measurement process in a position-based visual servo structure can be decomposed into two nonlinear transformations. First, the transduction and feature extraction functions, or world-to-feature space transformation, can be viewed as the inverse of an ideal interpretation, in the absence of noise, according to: CH2152-7/85/000010662$01 .OO